Extracting 3D Facial Animation Parameters from Multiview Video Clips

synthetic face’s behaviors must precisely conform to those of a real one. However, facial surface points, being nonlinear and without rigid body properties, have quite complex action relations. During speaking and pronunciation, facial motion trajectories between articulations, called coarticulation effects, also prove nonlinear and depend on preceding and succeeding articulations. Performance-driven facial animation provides a direct and convincing approach to handling delicate human facial variations. This method animates a synthetic face using motion data captured from a performer. In modern computer graphics-based movies such as Final Fantasy, Shrek, and Toy Story, character motion designers used optical or magnetic motion trackers to capture markers’ 3D motion trajectories on a performer’s face. They can track only a limited number of markers without interference, however, and the dozen or so markers they can place on facial feature points only sparsely cover the whole face area. Therefore, to derive a vivid facial animation, animators must adjust for the uncovered areas. Other approaches, discussed in the “Related Work” sidebar, also present limitations in analyzing and synthesizing facial motion. To tackle this problem, we propose an accurate and inexpensive procedure that estimates 3D facial motion parameters from mirror-reflected multiview video clips. We place two planar mirrors near a subject’s cheeks and use a single camera to simultaneously capture markers’ front and side view images. We also propose a novel closed-form linear algorithm to reconstruct 3D positions from real versus mirrored point correspondences in an uncalibrated environment. Figure 1 shows such a reconstruction.